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Academic  research  on  computer  based  instruction  (CBI)  has  dealt 
largely  with  CBI  of  knowledge  systems,  coherent  bodies  of  essentially 
propositional  knowledge.  Little  research  has  been  performed  on 
techniques  for  effective  CBI  of  dynamic  skills,  those  amalgams  of 
perceptual,  motor,  and  decision-making  skills  that  are  required  by  many 
real-time  event-driven  tasks.  The  demands  of  dynamic  skill  training  on 
student  processing  resources  are  different  from  those  of  knowledge 
system  teaching.  These  differences  suggest  that  the  techniques  found 
to  be  effective  in  conventional  CBI  may  not  be  applicable  to  dynamic 
skill  training  CBI.  Two  classes  of  research  issues  to  be  explored  are 
techniques  for  presentation  of  simulation  practice  and  methods  for 
providing  effective  instructional  feedback. 

A  microcomputer-based  experimental  simulation  training  system  for 
research  on  dynamic  skill  training  is  described.  Experimental  subjects 
are  taught  to  perform  a  simulation  task  based  on  the  job  of  an  air 
intercept  controller.  The  training  program  permits  controlled 
differences  in  instructional  treatment  for  different  groups  of 
students,  in  order  to  explore  empirical  Issues  in  dynamic  skill 
training.  X/ 
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INTRODUCTION 


Ve  Intend  to  draw  a  distinction  between  the  teaching  of  knowledge 
systems  and  the  training  of  dynamic  skills.  By  "knowledge  systems"  we 
mean  to  refer  to  sets  of  related  facts  that  are  commonly  taught  as  a 
body  of  coherent  subject  matter.  Examples  of  knowledge  systems  include 
the  history  of  England  under  the  Plantagenate  kings,  the  geological 
features  of  Western  North  America,  and  the  multiplication  tables.  By 
"dynamic  skills"  we  mean  sets  of  intellectual  processes  responsible  for 
selective  perceptions  in  a  real  time  driven  context  and  for  the 
selection  and  perfomance  of  appropriate  responses  in  that  context. 
Examples  include  vehicular  control  skills,  such  as  driving  a  car  or 
piloting  an  aircraft,  and  the  observational  and  decision-making  skills 
of  an  air  traffic  controller.  The  teaching  of  knowledge  systems  is 
characteristic  of  pedagogy  in  schools,  while  dynamic  skills  training  is 
typical  of  many  instances  of  on  the  job  training.  Probably  as  a  result 
of  this  distribution  of  these  two  types  of  learning,  there  has  been 
more  research  on  the  teaching  of  knowledge  systems  in  academic  research 
in  education.  It  cannot  be  assumed,  however,  that  the  findings  of  this 
academic  research  will  apply  to  dynamic  skills  training. 

The  distinction  is  important  because  it  cannot  be  assumed  that 
those  techniques  which  have  been  found  to  be  effective  for  teachning 
knowledge  systems  will  also  be  effective  for  teaching  dynamic  skills. 
Dynamic  skill  training  typically  involves  Intense  student  Involvement 
in  the  practice  of  the  skill.  Under  these  conditions  it  cannot  be 
assumed  that  students  have  the  same  attentional  resources  available  to 
process  instructional  messages*  Dynamic  skill  training  typically 
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involves  practice  sessions  in  which  students  must  learn  how  to  allocate 
resources  to  the  tasks  of  global  and  local  planning.  Unlike  typical 
knowledge  system  training,  dynamic  skill  training  imposes  inflexible, 
strict  time  limitations  on  student  responses  to  events.  Cognitive 
processing  differences  in  the  nature  of  the  learning  processes  suggest 
that  effective  teaching  of  dynamic  skills  requires  different  methods 
than  effective  teaching  of  knowledge  systems.  The  research  program 
described  here  is  designed,  in  part,  to  test  this  hypothesis. 

Research  on  computer-based  instruction  and  training  (CBIT)  has 
emphasized  techniques  and  principles  for  teaching  knowledge  systems. 
There  are  a  number  of  reasons  for  this  emphasis.  Most  CBIT  course 
implementations  were  developed  to  convey  knowledge  systems,  and  these 
existing  courses  have  provided  convenient  test  beds  for  research.  In 
addition,  many  researchers  are  inclined  by  both  training  and  experience 
to  focus  on  the  transmittal  of  knowledge  systems  such  as  those  taught 
in  academic  school  and  university  courses.  Knowledge  systems  are  not, 
however,  the  only  class  of  subject  matter  for  CBIT.  A  number  of 
computer-based  training  systems  are  in  use  whose  function  is  not  to 
teach  a  knowledge  system,  but  rather  to  convey  a  dynamic  skill  to  the 
students.  In  particular,  a  significant  financial  committment  has  been 
made  to  dynamic  skill  training  in  the  field  of  airplane  and  helicopter 
simulators  for  use  in  training  pilots.  These  simulators  can  be 
expected  to  be  made  more  effective  as  a  result  of  basic  research  on 
dynamic  skill  training  in  CBIT.  Yet  the  training  of  dynamic  skills 
has  thus  far  received  little  attention  in  the  academic  community. 

For  the  most  part,  the  CBI  research  community  has  dealt  only  with 
the  teaching  of  knowledge  systems.  Whole  books  have  been  written  on 
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CAI/CBI  (Bur son,  1976;  Taylor,  1980;  and  O'Neil,  1981,  ror  example) 
without  even  mentioning  dynamic  skill  training  in  CBIT  (although  others 
such  as  Hickey,  1975,  have  dealt  with  the  topic  of  simulation 
training).  There  is  clearly  a  need  for  basic  research  in  this 
important  training  area.  There  is  scope  for  considerable  development 
in  training  systems  to  teach  dynamic  skills.  Vehicular  control  skills, 
certain  production  control  systems  in  industry,  and  many  military  jobs 
(such  as  ground  controlled  landing  controllers  and  air  intercept 
controllers)  require  combinations  of  perceptual,  motor,  and  decision 
making  skills.  These  training  opportunities  can  be  productively 
addressed  by  computer  based  training  systems,  but  there  is  reason  to 
believe  that  the  challenge  cannot  be  met  only  with  the  pedagogical 
techniques  and  theories  developed  on  the  basis  of  research  on  CBI  for 
knowledge  systems.  The  attentional  demands  of  dynamic  skill  simulation 
training,  for  example,  may  make  conventional  approaches  to 
Instructional  feedback  to  student  actions  Ineffective. 

The  research  community  now  has  an  opportunity  to  conduct  a  new 
course  of  research  on  CBIT  of  dynamic  skills.  There  are  several 
reasons  to  expect  this.  Dynamic  skill  training  typically  implies  fast 
real-time  processing,  often  of  analog  inputs  from  the  student/trainee. 
Conventional  CAI  implementations  on  large  time-sharing  computers,  tied 
to  remote  terminals  limited  to  a  300  to  1200  baud  data  rate  over 
telephone  lines,  are  not  conducive  to  the  development  of  courses  on 
real-time  dynamic  skills.  Many  such  dynamic  tasks  may  require 
processor  dedication  to  ensure  prompt  handling  of  interrupt  conditions 
and  other  real-time  aspects  of  the  task.  Recent  advances  in  hardware 
and  systems  software  will  now  make  such  training  systems  more 


economical.  On  the  hardware  side,  inexpensive  microcomputer  systems 
make  single-user  CBIT  systems  financially  feasible.  On  the  software 
side,  the  development  of  a  number  of  microcomputer  operating  systems 
that  support  faster  executing  and  more  structured  high  level  languages 
than  were  formerly  available  make  dynamic  simulation  programming  on 
these  systems  both  feasible  and  maintainable. 

Increased  feasiblity  of  research  on  dynamic  skill  training  is  not 
the  only  reason  to  expect  increased  efforts  in  this  area.  There  is  a 
growing  demand  for  practical  CBIT  systems  in  this  area  for  training 
skills,  such  as  vehicular  or  other  machine  system  control,  that  require 
the  fusion  of  perceptual,  motor,  and  decision-making  skills.  In 
particular,  the  military  services  require  superior  training  methods  to 
accomplish  these  tasks,  given  the  decreased  level  of  education  and 
training  of  entry-level  personnel  in  the  all-volunteer  military. 

There  is  an  extensive  history  of  simulation  training  (or,  at 
least,  computer-based  simulation  drill  and  practice)  in  the  area  of 
flight  training.  The  development  of  these  simulation  drill  systems, 
frequently  at  very  great  expense  in  relation  to  the  projected  costs  of 
the  new  generation  of  dynamic  skill  training  systems,  was  motivated  by 
compelling  considerations  of  safety  and  economy.  As  the  complexity  of 
other  vehicular  systems  (such  as  military  tanks)  increases,  and  as  the 
cost  of  fuel  for  such  systems  continues  to  climb,  it  can  be  expected 
that  there  will  be  an  increased  demand  for  other  vehicular  skill 
simulators.  Maximum  effectiveness  will  be  obtained  from  these  training 
systems  if  they  are  not  used  merely  as  imitation  environments,  but  also 
provide  other  instructional  features.  For  example,  a  simulator  that 
can  provide  tutorial  interactions  in  addition  to  imitating  the  skill 
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environment  should  be  able  to  reduce  training  time  and  increase  its 
effectiveness  by  drawing  the  student fs  attention  to  individual  aspects 
of  the  simulation  or  his  responses  to  it. 

In  addition  to  vehicular  control,  other  complex  tasks  call  for  the 
development  of  economical  and  effective  CBIT  of  dynamic  skills.  For 
example,  the  Naval  Training  Equipment  Center  has  contracted  for  the 
development  of  experimental  innovative  CBIT  systems  to  train 
Ground-Controlled  Landing  Specialists  and  Air-Intercept  Controllers. 
These  jobs  require  that  the  worker  not  exercise  dr  e  t  vehicular 
control,  but  rather  assist  the  pilot.  This  is  accomplished  through 
observations  relevant  to  the  successful  control  of  aircraft  and  by 
transmitting  information  and  advice  to  the  pilots  of  the  aircraft. 

These  real-time  driven  tasks  require  the  acquisition  of  dynamic 
skills  for  successful  performance.  It  is  to  be  expected  that  many  tasks 
that  call  for  dynamic  man-machine  interactions  could  also  benefit  from 
operator  training  in  a  CBIT  simulation  system.  The  demand  for  such 
systems  can  be  expected  to  grow  when  CBIT  of  dynamic  skills  adds  other 
training  techniques  to  drill  and  practice,  and  presents  evidence  that 
it  can  make  effective  use  of  more  advanced  pedagogical  techniques. 

Our  laboratory  is  currently  engaged  in  a  program  of  research  on 
the  nature  of  dynamic  skill  acquisition  and  effective  approaches  to 
dynamic  skill  training  in  CBIT.  Under  funding  from  the  Office  of  Naval 
Research<1>,  a  low-cost  experimental  data  collection  system  for  research 
on  such  task  training  has  been  developed.  Our  project  has  four  goals: 

1.  Find  pedagogical  principles  for  CBIT  of  dynamic  skills. 


There  is  a  substantial  body  of  research  on  effective  pedagogy  for 
conventional  CBI  of  knowledge  systems.  It  is  likely  that  many  of  the 
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findings  of  this  research  will  also  apply  to  instruction  of  dynamic 
skills.  It  is  also  likely,  however,  that  significant  departures  from 
conventional  approaches  will  be  shown  to  be  beneficial.  For  example, 
it  is  likely  that  classic  findings  on  the  efficacy  of  immediate 
feedback  (knowledge  of  results)  will  not  be  upheld  for  highly  demanding 
real-time  dynamic  tasks,  in  which  the  intrusion  of  results  information 
would  be  expected  to  disrupt  the  training  task. 

2.  Model  cognitive  processes  in  dynamic  skill  exercise. 

CBIT  systems  for  dynamic  skills  instruction  will  provide  an  important 
source  of  data  for  psychologists  interested  in  modeling  the  cognitive 
processes  that  underly  dynamic  tasks.  A  CBIT  system  based  on  a 
dedicated  processor  can  record  a  wealth  of  data  about  a  student's  or  an 
expert’s  actions  during  a  simulated  task.  This  facility  can  be  used  to 
explore  cognitive  issues  in  dynamic  skills,  including  the 
representation  of  skill  components,  using  data  obtained  from  student 
training  with  the  experimental  CBIT  simulation  system. 

3.  Study  the  acquisition  of  complex  skills. 

The  acquisition  and  integration  of  the  skill  components  found  in 
complex  natural  tasks  probably  cannot  be  studied  economically  except 
through  data  acquired  from  a  dynamic  skill  CBIT  system.  A  study  of  the 
acquisition  of  vehicular  control  skills,  for  example,  could  be 
conducted  in  a  much  more  controlled  fashion  through  the  use  of  a 
simulator  than  through  observation  of  a  student  in  a  real  vehicle, 
k.  Study  the  conseauences  of  voice  I/O  in  simulation  training. 

Modern  dynamic  simulation  trainers  will  make  increasing  use  of  the 
new  technologies  for  speech  understanding  and  production.  These 
features  are  likely  to  significantly  modify  the  students'  perceptions 
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of  the  responsiveness  and  Intelligence  of  the  computer-based  systems 
with  which  they  are  interacting.  For  example v  voice  input  to  the 
system  may  contribute  to  an  unconscious  expectation  on  the  part  of  the 
student  that  the  system  or  some  simulated  component  of  the  system  will 
behave  in  a  very  human  manner.  If  a  CBIT  system  includes  a  simulated 
human  fighter  pilot  with  whom  the  student  communicates  by  voice,  then 
the  student  may  be  more  inclined  to  expect  the  "pilot"  to  understand 
deviations  from  the  presecribed  voice  responses  than  if  communication 
were  restricted  to  keyboard  console  interactions.  The  pedagogical 
consequences  of  such  perceptions  may  be  beneficial  or  detrimental,  but 
they  will  undoubtedly  require  careful  study. 

To  work  toward  these  goals,  we  liave  developed  an  inexpensive 
experimental  data-collection  system  for  research  on  computer-based 
training  of  real- time-driven  tasks.  The  system  makes  use  of  standard, 
off-the-shelf  microcomputer  products.  It  employs  the  UCSD  Pascal 
operating  system,  which  provides  many  times  the  execution  speed  of 
interpreted  BASIC  and  more  readable  and  maintainable  code  than  would 
result  from  the  use  of  BASIC,  Assembly  Language,  or  Forth.  The  system 
includes  more  than  8000  lines  of  Pascal  source  code  in  experimental 
simulation  and  training  programs  developed  in  our  facility.  These 
programs  were  developed  to  permit  the  implementation  of  a  range  of 
pedagogical  strategies,  so  that  experiments  with  different  pedagogical 
approaches  can  be  performed.  In  addition,  the  programs  have  built-in 
record-keeping  functions  to  keep  track  of  student  interactions  with  the 
simulation-training  system. 
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TEACHING  DYNAMIC  SKILLS 

An  ExflPPie  gf  a  Dynamic  Task 

In  order  to  facilitate  understanding  of  the  dynamic  task  training, 
an  example  of  a  dynamic  task  is  presented  here.  This  is  an  artificial 
task  developed  for  use  in  our  first  series  of  experiments  on  dynamic 
skill  training  in  CBIT.  The  task  is  a  moderately  complex  simulation 
game,  called  Air  Intercept  Controller.  It  is  based  on  the  task  of  a 
Navy  Air  Intercept  Controller,  and  includes  many  of  the  features  of  the 
real  task. 

The  student  plays  the  role  of  a  ground-based  Air  Intercept 
Controller.  Figure  1  shows  the  student  training  environment.  He  or 
she  is  seated  in  front  of  two  display  screens,  with  access  to  a 
keyboard  of  special  function  keys  and  a  joystick.  The  screen  on  the 
right  serves  as  a  simulated  radar  display.  Students  observe  the 
appearance  of  points  of  light  called  blips  on  the  screen  which 
represent  aircraft  in  a  defined  airspace.  Once  every  eight  seconds,  a 
"radar  sweep"  is  completed  and  the  blips  are  repositioned  on  the 
display.  The  second  display  screen  and  its  special  function  keyboard 
serve  as  a  simulated  tactical  data  system  console.  The  student  uses 
this  console,  together  with  the  associated  joystick,  to  acquire 
information  about  the  speeds  and  headings  of  the  aircraft  represented 
by  the  blips  on  the  screens.  The  student  presses  special  function  keys 
on  the  console  keyboard  to  send  directions  to  the  student* s  fighter 
aircraft.  These  directions  include  sending  appropriate  intercept  and 
attack  headings  so  as  to  direct  the  fighter  aircraft  to  correctly 
approach  the  enemy  aircraft.  When  the  student  judges  that  a  fighter  is 
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within  firing  range  of  an  enemy  craft,  he  or  she  can  direct  that  a 
missle  be  fired  at  it.  The  student  wins  a  game  when  his  aircraft 
survive  and  and  all  the  enemy  aircraft  are  shot  down.  The  task 
proceeds  at  a  rapid  pace.  Even  expert  students  are  hard-pressed  to 
perform  all  the  necessary  actions  in  order  to  attack  and  defeat  a 
number  of  the  enemy  aircraft  in  the  advanced  problems. 

The  experimental  training  system,  unlike  many  much  more  elaborate 
and  expensive  simulator- trainers,  such  as  those  commonly  used  in  flight 
training,  is  designed  to  do  more  than  to  provide  simulated  practice.  It 
can  also  interact  with  the  student  on  an  instructional  level.  In 
addition,  a  detailed  record  of  the  student’s  interactions  with  the 
system,  including  a  record  of  the  errors  made,  is  maintained.  This 
record  can  serve  as  a  basis  for  post  hoc  tutorials  and  replays. 

Although  the  experimental  training  system  was  developed  as  a 
research  tool,  it  is  likely  that  inexpensive  training  systems  such  as 
this  will  play  a  useful  role  in  many  training  applications.  A 
microcomputer-based  simulator  might  offer  less  superficial  fidelity 
than  the  real  environment  or  than  a  more  elaborate  simulator,  and  yet 
offer  more  effective  training.  A  simulator  system  that  includes 
facilities  for  tutorial  and  other  instructional  interactions  combined 
with  simulation  may  prove  to  be  a  more  powerful  instructional 
environment  than  even  the  most  realistic  simulation  system  that  lacks 
an  instructional  component.  Such  training-simulators  can  be  used  to 
teach  conceptual  aspects  of  real-time  tasks,  to  develop  subskills 
called  for  by  these  tasks,  or  simply  to  teach  essential  safety 
procedures  before  high  fidelity  training  commences.  A  training  system 
incorporating  this  philosophy  would  make  extensive  use  of  low-fidelity 
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simulation  to  accomplish  particular  training  objectives,  and  then 
polish  the  training  with  practice  on  real  systems  or  on  higher  fidelity 
(and  therefore  more  expensive)  simulator- trainers. 

Issues  in  the  Training  of  Dynamic  Skills 

There  are  undoubtedly  a  large  number  of  important  issues  to  be 
resolved  about  effective  approaches  to  CBIT  of  dynamic  skills.  Our 
research  group  is  concentrating  currently  on  two  classes  of  training 
issues:  presentation  strategies  and  knowledge-of-results  strategies  for 
effective  training.  The  dynamic  nature  of  the  training  under 
discussion  implies  a  wealth  of  presentation  options  not  found  in 
conventional  computer  based  instruction.  The  relative  merit  of  these 
different  approaches  to  presenting  simulation  training  should  be 
explored.  A  specific  major  training  issue  to  explore  is  to  isolate  the 
most  effective  means  for  conveying  to  the  student  or  trainee  the 
quality  of  his  or  her  responses  to  the  simulation.  It  is  expected  that 
the  findings  of  conventional  CBIT  for  giving  knowledge-of-results 
during  knowledge  system  teaching  will  not  apply  directly  to  dynamic 
skill  training.  If  they  do  not,  then  how  can  we  give  effective 
performance  feedback  to  the  students? 

Presentation  Strategies 

The  dynamic  nature  of  the  simulation  training  being  considered 
here  means  that  there  is  a  controllable  rate  at  ?hich  events  occur  in 


the  simulation.  Changes  in  the  simulated  environment  can  take  place 
more  or  less  rapidly,  or  even  halt,  under  program  control,  allowing  the 
student  to  process  some  aspect  of  an  ordinarily  dynamic  process  in  a 
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static  Banner.  The  range  of  options  dealing  with  changes  In  the  pacing 
of  events  we  call  event  rate  manipulations.  A  second  class  of 
presentation  strategies  are  those  which  permit  repetition  of  simulated 
episodes  or  episode  segments.  These  we  refer  to  as  repetition 
manipulations.  A  third  class  of  strategies  are  partial-task 
manipulations. 

Event  rate  manipulations 

Event  rate  manipulations  include  stop  action,  slow  motion,  and 
fast  action  episode  segments.  The  effectiveness  of  each  of  these 
techniques  must  be  explored  in  a  variety  of  simulation  environments. 

For  example,  if  slow  motion  simulation  has  a  beneficial  training  effect 
in  comparison  with  natural  speed  simulation  in  any  context,  one  would 
expect  to  observe  the  favorable  result  in  simulated  contexts  of  rapid 
pacing  that  impose  significant  processing  demands  on  the  student.  In 
addition,  it  is  to  be  expected  that  there  will  be  an  effect  due  to 
locus  of  control  (program  or  student  initiation)  of  these  options. 
Whether  students  can  employ  these  features  to  their  benefit  in  training 
is  a  matter  for  empirical  determination. 

Stop  action.  During  interactive  training  episodes,  students 
sometimes  feel  that  they  just  need  a  second  or  two  to  stop  and  think, 
in  order  to  assimilate  something  they  have  just  encountered  or  to  plan 
a  response.  A  training  system  that  provides  a  student-controlled 
"stop"  feature,  perhaps  in  the  form  of  a  menu  choice  or  a  special 
"stop”  key  on  the  keyboard,  would  give  students  in  dynamic  simulation 
training  such  an  opportunity  to  exercise  some  control  over  the  pace  of 
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events.  Use  of  such  an  option  would  often  mean  that  a  student  was 
using  a  conscious  decision-making  approach  to  the  task  at  hand  rather 
than  simply  reacting  In  an  automatic  manner  to  the  Information 
displayed.  Whether  a  problem-solving  orientation  to  the  tasks  of  a 
dynamic  skill  is  an  effective  method  for  acquiring  that  skill  is  an 
empirical  issue.  It  may  be  that  students  trained  with  this  method  will 
perform  less  competently  in  the  field  because  they  become  dependent  on 
the  use  of  the  slower-paced  problem  solving  method  encouraged  by  the 
training  system.  On  the  other  hand,  it  is  possible  that  students  would 
use  the  stop  action  feature,  and  the  problem-solving  approach  it 
encourages,  only  as  a  stepping-stone  to  automatic  competence.  If  this 
is  the  case,  the  stop  action  feature  may  promote  learning,  particularly 
among  less  practiced  students.  Experiments  are  called  for  to  resolve 
this  issue. 

Stop  action  may  also  prove  an  effective  instructional  device  under 
program  control.  During  expository  simulation,  the  task  could  be 
halted  to  direct  the  student’s  attention  to  some  feature  of  the 
simulation.  Instructional  feedback  may  benefit  from  the  use  of  stop 
action.  Because  students’  information  processing  loads  are  likely  to 
be  quite  high  during  the  training  of  difficult  tasks,  the  additional 
processing  load  imposed  by  instructional  feedback  is  likely  to  be 
Intellectually  overwhelming  in  a  continuous  simulation.  By 
Interrupting  the  simulation  during  a  feedback  presentation,  some 
student  processing  resources  that  were  dedicated  to  tracking  the 
simulation  should  become  available  for  attending  to  the  Instructional 
feedback. 
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Slow  Motion  and  speeded  motion.  The  participant  in  a  real-time 
event-driven  task  often  senses  that  the  action  is  beginning  to  get 
ahead  of  his  or  her  ability  to  keep  up  with  it.  Shortly  thereafter, 
the  action  is  likely  to  outpace  the  student,  and  a  catastrophic  decline 
in  performance  will  result.  This  kind  of  time-driven  student  failure 
may  or  may  not  have  desirable  pedagogical  consequences.  Such  failures 
may  motivate  students  to  automatize  their  processing  so  as  to  acheive 
more  responsiveness  tn  the  task.  On  the  other  hand,  these  episodes  may 
be  confusing  and  discouraging  for  students.  If  time-driven  failures 
are  not  pedagogicall>  productive,  an  optional,  student-controlled 
slow  motion  mode  might  prove  useful.  In  the  case  of  the  Air  Intercept 
Controller  simulation  described  above,  slow  motion  would  mean  that  the 
radar  screen  would  update  at  longer  intervals  than  normal,  and  the 
blips  on  the  screen  would  appear  to  move  more  slowly  than  in  the  normal 
speed  simulation.  Use  of  optional  slow  motion  would  lead  to  several 
logistic  problems  for  the  training.  One  would  be  to  provide  a  means 
for  returning  to  normal  presentation  rate.  If  this  is  under  student 
control,  we  may  find  that  students  prefer  to  spend  too  much  time  in 
slow  motion  mode  and  actually  retard  their  acquisition  of  the  skills. 
However,  if  the  simulation  simply  returns  to  normal  rate  after  an 
arbitrarily  short  period  of  time,  the  student  may  still  be  in  an 
overloaded  state.  The  best  long-term  solution  to  this  dilema  lie  in 
the  development  of  an  intelligent  student  monitor  routine  that 
estimates  the  student’s  overload  state  on  the  basis  of  the  speed  and 
appropriateness  of  his  responses  to  the  simulation.  This  monitor  would 
then  have  control  over  the  return  to  normal  speed.  In  fact,  such  a 
monitor  could  be  used  to  put  the  simulation  into  a  reduced  speed  mode 


when  it  perceived  a  need  to  reduce  the  student's  processing  load 


In  addition  to  use  as  a  pacing  device  during  practice  of  dynamic 
skills  on  simulators,  slow  motion  may  prove  to  be  an  effective 
instructional  presentation  tool.  When  a  complicated  new  series  of 
actions  is  to  be  taught  to  a  student,  he  or  she  could  be  required  to 
step  through  the  sequence  at  slower  than  natural  rates  the  first  few 
times. 

Ideally,  the  rate  of  presentation  of  the  simulation  should  be 
continuously  variable.  If  a  sophisticated  student  monitor  can  evaluate 
a  student's  momentary  processing  load  with  some  accuracy,  then  the 
simulted  rate  of  events  could  be  slowed  down  or  speeded  up  to  match  the 
student's  present  ability  to  handle  the  incoming  data.  One  problem 
with  this  approach  is  that  the  student  may  actually  be  getting 
misinformation  about  the  task.  If  one  of  the  things  a  student  learns 
in  acquiring  a  dynamic  skill  is  the  natural  pace  of  events,  then  an 
inconsistent  simulation  pace  may  not  give  the  student  a  realistic  view 
of  the  real  world  task.  Only  empirical  study  can  reveal  whether 
dynamic  task  training  benefits  from  slow  motion  or  fast  motion 
presentations. 

Repetition  Manipulations 

Action  reolavs.  In  an  action  replay,  a  portion  of  a  simulated 
task  is  repeated  in  order  to  provide  a  student  with  repeated  practice 
on  the  elements  of  the  skill  required  by  that  portion.  One  potential 
application  of  action  replays  is  as  a  method  for  instructional 
feedback.  When  the  system  determines  that  performance  on  a  portion  of 
a  practice  task  was  not  acceptable,  it  can  require  of  the  student  that 
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that  portion  be  repeated.  This  form  of  simulation  instruction  borrows 
a  simple  looping  technique  from  conventional  computer  based 
instruction  in  which  students  repeat  a  module  until  they  meet  some 
performance  criterion. 

Another  possible  use  of  action  replays  is  under  student  control. 
Sometimes  after  completing  a  short  sequence  in  a  simulated  event  a 
student  feels  that  he  or  she  then  understands  how  the  sequence  should 
have  been  done.  If  the  student  is  required  to  continue  with  the  next 
portion  of  the  task  immediately,  the  insight  may  be  lost,  and,  in  any 
case,  there  may  be  no  immediate  opportunity  to  apply  it.  If  the 
student  has  the  option  to  replay  the  sequence,  applying  a  new 
understanding  of  what  should  be  done,  this  extra  practice  could  give 
the  student  a  chance  to  consolidate  his  or  her  understanding.  At  the 
end  of  the  replayed  segment,  the  simulated  action  would  continue  as 
though  the  student  had  made  those  responses  on  the  original  segment. 

Such  action  replays  would  allow  students  to  practice  short 
sequences  of  actions  at  times  when  they  believe  the  practice  will  be 
useful.  There  may  be  positive  motiviational  effects  from  such  replays, 
since  they  would  allow  students  to  avoid  simulated  catastrophes,  if 
they  observe  that  such  a  consequence  is  about  to  occur  as  a  result  of 
an  error.  Students  could  be  expected  to  use  the  feature  to  drill  on  a 
sequence  representative  of  new  skill  demands  until  they  believe  they 
have  attained  mastery.  A  possible  negative  consequence  of  the 
availability  of  this  option  is  that  some  students  may  overuse  the 
option,  as  a  result  of  unrealistically  high  performance  criteria.  Too 
much  time  could  be  spent  in  training  of  relatively  simple  procedures  if 
students  have  total  control  over  pacing  through  unlimited  use  of  the 
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action  replay  feature.  Teats  of  the  use  of  this  dynmaic  skill  training 
feature  may  therefore  reveal  a  need  for  the  system  to  ration  use  of 
this  mode  of  training. 

Partial-task  Manipulations 

In  the  past,  computer-based  simulator  trainers  for  dynamic  skills 
training,  as  developed  for  flight  training  and  military  systems 
training,  have  been  sophisticated  and  very  expensive  devices  capable  of 
simulating  very  realistically  almost  the  whole  range  of  activities  that 
could  be  performed  with  the  real  world  system.  Visual,  auditory,  and 
often  proprioceptive  stimuli  are  provided  by  these  elaborate 
simulators.  The  increasing  availability  of  inexpensive  stand-alone 
microcomputer  systems  suggests  that  it  may  be  desirable  to  determine  to 
what  extent  portions  of  the  training  task  can  be  performed  by  simpler 
simulation  devices. 

One  approach  to  more  cost  effective  training  might  be  to  make  use 
of  trainers  that  are  not  designed  to  simulate  the  whole  of  the  task  to 
be  learned,  but  only  some  portion  of  that  task.  For  example,  decision¬ 
making  components  of  an  air-intercept  controller  task  could  be  taught 
separately  from  information-acquisition  skills  and  from  communications 
skills.  The  processes  of  information  acquisition,  in  particular,  call 
for  a  high  degree  of  motor  skills  in  this  task.  It  is  possible  that 
some  basic  decision-making  skills  and  resource-allocation  skills  could 
be  taught  without  making  use  of  a  completely  accurate  representation  of 
a  real  air  Intercept  controller  environment,  using  a  simpler  but 
functionally  related  simulated  task.  Experiments  are  called  for  to 
determine  whether  there  is  a  transfer  of  training  from  a  primitive 
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simulation  to  performance  on  a  morji  sophisticated  simulator  or  to 

performance  on  the  real  task.  Part  task  training  research  has  yielded 

mixed  results.  In  some  cases,  tfie  partial  tasks  teach  response 

patterns  which  are  antagonistic  to  those  required  in  the  transfer  task. 

In  such  cases,  part-task  training  can  have  a  negative  impact  on 

performance  on  the  target  task.  Part- task  training  is  most  often  shown 

to  be  effective  for  cognitive,  decision-making  components  of  complex 

tasks,  rather  than  forHeaching  motor  responses.  These  findings 
> 

suggest  a  coursexof  research  for  part-task  training  of  dynamic  skills. 

In  addition  to  research  on  traditional  part-task  training 
£ 

approaches,  dynamic  skill  simulator  training  calls  for  research  on 


progressive  task 


?nt .  In  this  technique,  a  problem  or  problem 


type  is  successively  enlarged  or  complicated.  Students  are  taught  a 
few  simple  skills  and  are  given  simulated  practice  problems  that  are 
quite  natural,  but  are  unusual  in  that  they  require  only  the  already 
presented  skills.  Then  more  skills  are  taught  to  the  students  and 
somewhat  more  complex  practice  problems,  which  call  for  these  skills, 
are  used  for  practice.  This  technique  is  commonly  used  in  introductory 
computer  programming  classes.  In  this  environment,  students  are  taught 
only  portions  of  a  programming  language  at  a  time,  and  the  problems 
they  are  assigned  are  appropriate  for  that  portion  of  the  language  that 
they  know. 

Progressive  task  enlargement  offers  a  natural  approach  to 
incrementing  a  student* s  performance  level.  For  dynamic  skill 
training,  it  may  provide  a  method  for  manipulating  problem  difficulty 
without  artificially  manipulating  the  natural  rate  of  events  as  do  the 
other  event  rate  manipulation  techniques  discussed  above. 


Page  18 


Instructional  Eggflfe ask 

One  of  the  most  firmly  established  principles  of  learning  research 
is  the  importance  of  prompt  knowledge  of  results  following  student 
responses.  Feedback  about  the  correctness  of  a  student's  interactions 
is  not  provided  immediately  by  most  dynamic  skill  simulators.  Students 
can,  of  course,  often  deduce  the  appropriateness  of  their  responses  on 
the  basis  of  the  simulated  outcomes.  If  the  student  "pilot"  of  a 
simulator  takes  an  action  and  shortly  thereafter  is  told  that  the 
simulated  aircraft  has  crashed  and  burned,  then  the  student  may 
conclude  that  the  action  was  inappropriate  in  the  circumstance.  This 
type  of  "natural  consequences"  feedback  has  its  limitations,  despite 
the  powerful  impact  it  can  sometimes  provide.  One  such  limitation  is 
that  it  does  not  explicitly  pair  actions  and  consequences.  In  the  case 
of  the  simulated  crash  just  mentioned,  it  is  likely  that  the  pilot 
would  attribute  the  responsibility  for  the  simulated  accident  to  the 
last  non-emergency  action  he  took  .  This  attribution  could  be 
erroneous.  A  natural  consequences  approach  to  feedback  requires  that 
students  interpret  their  own  errors.  In  addition,  some  student  actions 
may  be  incorrect  or  inappropriate  and  yet  not  result  in  perceptible 
problems  most  of  the  time.  In  these  cases  the  student  will  not  get 
feedback  about  the  incorrect  action  except  under  certain  conditions. 

Benefits  can  be  expected  from  providing  instructional  capabilities 
in  dynamic  interactive  trainer  simulators,  such  as  the  air  intercept 
controller  discussed  above.  What  form  such  feedback  augmentations 
should  take  is  a  matter  for  basic  research.  Feedback  techniques  used 
for  knowledge  system  teaching  probably  cannot  be  straightforwardly 
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applied  to  training  real  time  driven  decision  making  tasks.  One  reason 
for  this  is  that  the  dynamic  simulation  acquires  its  realism  in  part 
form  the  ongoing,  episodic  nature  of  the  simulated  events.  Feedback 
(other  than  natural  consequences)  serves  to  interrupt  the  natural 
episode,  and  to  distract  the  student  from  the  simulated  event.  When 
knowledge  system  training  is  the  goal  of  a  computer-based  simulation, 
such  interruptions  are  no  problem.  Attention  can  be  briefly  diverted 
and  then  returned  to  the  evidence  at  hand.  In  many  cases  (e.g.,  the 
BTL  Trainer  Simulator  —  Rigney,  Towne,  Moran,  &  Mishler,  1978; 

Munro,  Towne,  &  King,  1980;  Towne  &  Munro,  1981),  the  student  can 
replay  portions  of  the  simulation  in  order  to  reconstruct  his  findings 
if  his  memory  is  disrupted.  Such  an  approach  is  not  ordinarily 
possible  in  the  simulation  of  a  dynamic  event. 

The  experimental  system  described  below  has  been  developed  in  part 
to  explore  the  pedagogical  consequences  of  two  aspects  of  the 
presentation  of  feedback  information  in  dynamic  skill  training. 

The  first  aspect  is  the  continuity  of  the  simulation  during  feedback 
episodes.  The  second  aspect  of  feedback  presentation  to  be  studied  is 
student  control  over  when  feedback  will  be  presented,  a  factor  we  will 
refer  to  as  feedback  intrusiveness. 

In  order  to  anticipate  the  impact  of  either  simulation  continuity 
or  feedback  intrusiveness  upon  training  effectiveness  we  need  to 
consider  the  demands  which  are  placed  upon  the  cognitive  resources  of 
the  student  when  confronted  with  a  dynamic  skills  training  task.  Two 
primary  types  of  cognitive  skill  are  likely  to  be  affected  by  these 
training  task  variables.  First,  during  a  dynamic  skills  training  task 
the  student's  attentional  resources  are  often  heavily  taxed.  Students 
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oust  attend  to  the  flow  of  events  in  the  simulation,  and  they  must 
attend  to  other  essential  training  information,  such  as  feedback. 

Second,  students'  encoding  and  representational  skills  are  exercised  to 
produce  accurate  cognitive  representations  of  the  task  and  its 
structure,  in  order  to  produce  immediately  required  responses  and  to 
plan  for  future  contingencies.  The  factors  of  simulation  continuity 
and  feedback  intrusiveness  must  be  evaluated  in  terms  of  their  impact 
upon  attentional  and  representational  skills. 

Simulation  Continuity 

Consider  the  following  two  strategies  for  simulation  continuity 
during  administration  of  instructional  feedback;  the  training  simulation 
can  be  stopped  until  the  feedback  is  complete  or  it  can  be  allowed  to 
continue  normally  during  presentation  of  the  instructional  feedback. 
Consideration  of  the  cognitive  factors  just  mentioned  ~  attentional 
monitoring  and  representation  formation  —  seem  to  lead  to  contrary 
points  of  view  about  the  relative  effectiveness  of  these  two  strategies 
for  presenting  the  simulation. 

With  respect  to  the  attentional  processes  of  the  student,  the 
interrupted  simulation  strategy  has  the  potentially  beneficial  feature 
of  reducing  the  attentional  load  during  feedback  episodes  by 
eliminating  the  need  to  attend  to  the  simulation  when  feedback 
information  is  being  presented.  In  the  case  of  a  continuously 
presented  simulation,  a  student's  processing  capacity  is  likely  to  be 
largely  taken  up  with  attending  and  responding  to  the  simulation.  The 
addition  of  the  task  of  attending  to  the  instructional  message  while 
the  simulation  continues  is  likely  to  overload  attentional  mechanisms, 
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to  the  detriment  of  both  performance  on  the  simulation  task  and 
attention  to  the  feedback.  It  is  therefore  reasonable  to  expect  that 
performance  under  training  conditions  in  which  the  simulation  is 
interrupted  during  feedback  episodes  will  be  superior  to  performance 
when  presentation  of  the  simulation  is  continuous  throughout  the 
training  session. 

On  the  other  hand,  the  formation  of  a  precise  and  accurate 
representation  for  a  task  in  which  there  are  real  time  elements  may  be 
hindered  by  frequent  interruptions  of  task  continuity.  It  is  reasonable 
to  expect  that  a  complete  mental  representation  of  a  task  involving 
coordination  of  responses  to  real  time  events  must  contain  some  accurate 
portrayal  of  the  temporal  relationships  among  the  various  task  elements. 
That  is,  if  one  of  the  things  students  learn  is  the  rate  at  which  a 
series  of  simulated  events  is  to  occur,  then  the  interruption  of  this 
pacing  may  be^disruptive  to  the  representational  mechanisms.  There  is  no 
direct  experimental  evidence  for  such  representational  mechanisms,  at 
least  in  the  context  of  training.  However,  the  studies  by  Shepard  and 
his  colleagues  (Shepard  &  Metzler,  1971;  Cooper  &  Shepard,  1978)  suggest 
that  mental /imaginal  processes  can  provide  detailed  representations  for 
certain  types  of  continuously  variable  physical  events  such  as  rotation 
of  a  physical  object.  It  is  an  open  question  whether  such  representational 
abilities  extend  to  the  diversity  of  temporal  relationships  which  exist 
in  a  typical  dynamic  skills  task.  In  addition,  it  is  possible  that 
a  student  could  overcome  the  disruptive  effects  of  interrupting  the 
simulation  by  performing  some  kind  of  "mental  subtraction"  of  any  feedback 
time  which  Intervened  between  two  simulation  events  to  be  temporally 
related.  Therefore,  it  is  possible,  but  by  no  means  certain,  that 
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Interrupting  the  simulation  during  feedback  episodes  will  lntefere  with 
the  student's  ability  to  develop  a  complete  and  accurate  representation 
of  the  simulated  task. 

Feedback  Intrusiveness 

The  second  research  issue  in  instructional  feedback  for  dynamic 
skills  training  is  feedback  intrusiveness  and  its  effect  upon  a 
student's  attentional  and  representational  cognitive  processes. 

Feedback  intrusiveness  refers  to  whether  the  student  controls  the  time 
at  which  feedback  information  is  presented  in  a  training  task. 

Feedback  under  student  control  is  termed  non-intrusive  feedback f  and 
externally  paced  presentation  of  feedback  is  termed  intrusive.  This 
terminology  is  motivated  by  noting  an  analogy  of  student-training  system 
interaction  to  a  conversational  context. 

The  student's  interaction  with  the  dynamic  skills  CBIT  system  has 
parallels  to  a  conversational  interaction  between  two  people.  In 
normal  conversation,  the  participants  exchange  indications  of  their 
readiness  to  accept  input  from  each  other;  that  is,  they  signal 
turn-taking  in  the  conversation.  In  conventional  knowledge  system 
teaching  in  CBI,  each  student  response  (such  as  typing  in  the  answer  to 
a  question)  can  be  thought  of  as  a  signal  that  the  student  has 
surrendered  a  conversational  turn.  After  making  a  response,  the 
student  expects  a  reply  from  the  teaching  system.  In  dynamic  skill 
training,  however,  each  student  interaction  with  the  simulation  system 
is  not  a  conversational  turn  surrender*  Instead,  the  student  remains 
actively  engaged  with  the  simulation,  preparing  a  series  of  actions  in 
response  to  observed  and  expected  simulated  events.  In  this  context, 
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the  Interjection  of  an  Instructional  feedback  message  is  an  intrusion. 
The  system  is,  in  some  sense,  breaking  in  on  the  student's  turn. 

Consider  the  impact  upon  attentional  processing  of  providing 
feedback  by  intrusive  versus  non-intrusive  means.  During  a  complex 
training  session,  a  student's  processing  resources  are  likely  to  be 
largely  absorbed  in  attending  and  responding  to  the  task  simulation 
Itself.  If  the  task  is  suddenly  Intruded  upon  —  whether  or  not  the 
simulation  is  stopped  during  the  feedback  episode  —  the  Intrusion  is 
likely  to  require  additional  processing  resources  to  perform  the 
attentional  shift.  This  surge  in  processing  resource  demand  is  likely 
to  interfere  with  the  normal  learning  and  performance  processes.  If 
the  disruption  occurs  at  a  point  in  the  task  when  a  large  percentage  of 
cognitive  resources  are  already  committed,  then  either  or  both  of  the 
attention  to  the  feedback  and  task  performance  are  likely  to  suffer. 

The  student  will  presumably  shift  attention  from  the  task  simulation  to 
feedback  Information  only  at  points  where  processing  requirements  are 
relatively  low  so  that  the  attentional  shift  may  be  accomplished  with  a 
minimum  of  disruption  to  response  and  learning  processes.  If  the 
argument  just  given  is  sound  then  we  expect  that  feedback  given 
intrusively  will  be  more  disruptive  of  both  learning  and  performance 
than  will  feedback  provided  non-intrusively. 

These  points  about  attentional  processes  underscore  our  analogy 
between  conversation  and  student  interaction  with  the  training  system. 
It  seems  likely  that  conversational  cooperation  is  required  for  reasons 
quite  like  those  just  discussed  for  skills  training.  In  this  view, 
conversational  cooperation  is  an  adaptation  to  information  processing 
resource  limitations.  Therefore,  it  is  our  expectation  that  a  dynamic 
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skills  training  system  that  partially  emulates  some  of  the  features  of 
conversational  turn-taking  in  Instructional  feedback  will  prove 
superior  to  a  system  that  arbitrarily  interrupts  student  task 
processing  with  instructional  messages.  This  emulation  consists  of 
signaling  a  readiness  to  provide  feedback  in  a  non-intrusive  way,  and 
then  postponing  the  presentation  of  the  message  until  the  student 
explicitly  surrenders  a  turn  and  requests  presentation  of  feedback. 

It  is  plausible  to  claim  that  the  non-intrusive  feedback  condition 
(student  paced  presentation  of  feedback)  could  present  difficulties  for 
representational  (as  opposed  to  attentional)  processes.  This  is  due  to 
the  fact  that  the  student  will  frequently  wait  to  see  a  feedback 
message  until  some  time  after  the  context  to  which  the  message  refers 
has  passed.  The  traditional  results  on  delay  of  reinforcement  during 
learning  indicate  such  a  delay  will  be  detrimental  to  learning.  Thus, 
if  feedback  serves  as  a  reinforcer  (positive  or  negative)  the  delays 
which  occur  in  the  non-intrusive  conditions  should  impair  overall 
student  performance.  We  can  put  this  in  a  more  cognitive  perspective 
by  noting  that  when  the  feedback  message  is  delayed  it  should  be 
more  likely  that  the  content  of  such  a  message  will  not  be  correctly 
related  to  the  student’s  representation  of  the  task.  The  student  could 
readily  be  confused  as  to  the  context  to  which  the  feedback  message 
refers.  Thus,  we  have  two  types  of  argument  that  students  in  the 
non-intrusive  condition  will  be  at  a  disadvantage  in  making  use  of  the 
feedback  information.  The  students  whose  feedback  is  presented 
Intrusively  always  receive  the  feedback  message  immediately  following 
the  context  for  which  it  is  generated  and  so  should  not  suffer  this 
representational  and  reinforcement  disadvantage. 
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These  arguments  are  challenged  if  we  postulate  that  students  have 
some  ability  to  reinstate  the  context  to  which  a  feedback  message 
refers  from  the  content  of  that  message*  The  research  specifically 
addressed  to  delay  of  knowledge  of  results  tends  to  support  this  point 
of  view  (Kling  &  Schrier,  1971).  In  these  studies  it  appeared  that 
making  students  wait  for  feedback  was  not  detrimental  if  little  or  no 
responding  was  required  during  the  wait  interval.  A  cognitive 
interpretation  of  these  findings  is  that  students  were  (due  to  memorial 
processes)  able  to  maintain  a  trace  of  the  context  for  the  knowledge  of 
results  information.  If  students  in  a  dynamic  skills  training  task 
were  similarly  able  to  recall  the  context  of  a  feedback  message,  then 
it  would  follow  that  students  receiving  feedback  non-intrusively  would 
not  suffer  a  difficulty.  Therefore,  it  is  an  open  question  as  to  how 
the  variable  of  intrusiveness  will  affect  students  in  a  dynamic  skills 
task.  Attentional  demand  considerations  suggest  superior  learning 
performance  for  students  in  a  non-intrusive  feedback  condition. 
Representational  and  reinforcement  considerations  argue  for  the 
conclusion  that  students1  performance  in  the  non-intrusive  feedback 
condition  will  be  adversely  affected  unless  the  feedback  messages  they 
receive  are  sufficiently  clear  that  they  reinsatate  the  referent 
context. 
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A  Design  £flt  FttflfrftfiK  Research. 

The  table  below  repreaenta  a  two  factor  dealgn  for  studying  the 
effecta  of  simulation  continuity  during  feedback  and  feedback 
intrusiveness. 
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Six  different  conditions  for  the  presentation  of  instructional  feedback 
messages  are  presented  here: 

1 .  Intrusive  feedback  with  interrupted  simulation.  The  student  has  no 
control  over  presentation  of  the  feedback  messages.  When  the  system 
detects  a  student  error  or  a  condition  that  demands  student  attention, 
it  freezes  the  simulation,  sounds  an  audible  feedback  warning  tone  and 
presents  the  feedback  message  In  a  reserved  area  of  a  display  screen. 
The  simulation  remains  frozen  until  the  user  "accepts"  the  feedback  by 
depressing  a  special  key.  At  that  point  the  simulation  resumes. 

2#  Non-intrusive  feedback  with  interrupted  simulation.  When  the 
training  system  detects  a  student  error  or  other  condition  that 
warrants  instructional  feedback,  it  sounds  an  audible  feedback  warning 
tone.  The  simulation  continues  without  interruption.  When  the  student 
feels  ready  to  see  the  feedback  message,  a  special  "help"  key  is 
depressed.  At  this  point  the  simulation  freezes  and  the  student  can 
read  the  message.  The  user  then  "accepts"  the  feedback  by  depressing 
another  special  key.  At  that  point  the  simulation  is  unfrozen  and 
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continues  normally. 

3.  Intrusive  feedback  with  continuous  simulation.  In  this  mode  of 
Instruction,  the  system  presents  feedback  messages,  as  in  1 ,  whenever 
their  conditions  occur.  The  simulation  is  not  interrupted,  however, 
during  presentation  of  the  instructional  message. 

4.  Non-intrusive  feedback  with  continuous  simulation.  The  system 
signals  the  availability  of  feedback  messages,  but  presents  them  only 
upon  student  request,  as  in  2.  As  in  3,  the  simulation  is  not 
interrupted  during  feedback  presentation. 

5.  Intrusive  feedback  with  continuous  but  invisible  simulation.  As  in 

1  and  3,  the  instructional  messages  are  presented  whenever  their 
conditions  arise.  The  simulation  screen  goes  blank  for  the  student 
during  the  feedback.  When  feedback  ends,  the  simulation  becomes 
visible  again,  but  in  the  state  to  which  it  advanced  during  the 
feedback  presentation. 

6*  Non-intrusive  feedback  with  continuous  invisible  simulator.  As  in 

2  and  4,  instructional  messages  are  presented  at  student  request.  The 
simulation  continues  in  an  invisible  mode  until  the  student  accepts  the 
feedback  message. 

The  first  goal  of  our  research  project  is  to  explore  the 
consequences  of  different  combinations  of  the  simulation  continuity  and 
feedback  intrusiveness  factors  on  the  success  of  dynamic  skill 
training.  To  that  end,  an  experimental  test  bed  training  system  has 
been  developed.  The  features  of  this  system  are  described  in  the  next 
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THE  EXPERIMENTAL  SYSTEM 

Figure  2  presents  the  functional  structure  of  our  experimental 
dynamic  skill  training  system.  A  student  training  episode  is  viewed  as 
having  two  major  components.  The  first  is  a  pre-training  session,  in 
which  the  student  is  Introduced  to  the  task  and  to  the  activities  that 
comprise  the  skill.  The  second  is  a  practice  training  session,  in 
which  the  student  receives  a  combination  of  simulated  practice  and 
tutorial.  These  tutorial  components  are  not  separate  from  the  practice 
episodes,  but  are  intertwined  with  the  practice  activities.  One  of  the 
research  questions  being  pursued  with  the  system  is  how  to  most 
effectively  mix  practice  and  tutorial  instruction. 

A  complex  set  of  data  is  recorded  during  each  student  practice 
training  session.  This  data  set  includes  records  of  student  actions, 
of  the  occurence  of  certain  simulated  events,  of  detected  student 
errors,  and  of  tutorial  interactions  with  the  student.  A  set  of  data 
extraction  programs  are  applied  to  these  data  to  produce  data  relevant 
to  research  issues  of  interest.  These  extracted  data  are  then  analyzed 
with  the  SPSS  data  analysis  package  on  the  University  mainframe 
computer. 

The  Simulation  Trainer  Program 

The  current  version  of  the  dynamic  skills  trainer  program 
simulates  a  game-like  task  similar  to  that  of  an  air  intercept 
controller.  Students  are  trained  on  the  task  in  a  single  training 
session  of  approximately  four  hours,  training  consists  of  an 
Introductory  session  of  one  and  one  half  hours  in  which  the  student 
sees  viedeotaped  explanations  of  the  task  and  the  equipment  and  works 
through  an  Introductory  training  program.  This  Introductory  program 


requires  that  the  student  read  brief  explanations  of  each  of  the 


controls  available  to  the  student  on  the  simulated  control  console. 
After  reading  each  explanation,  the  student  is  required  to  briefly 
interact  with  a  simple  simulation  that  requires  the  exercise  of  the 
Just-discussed  function.  Students  who  fail  to  correctly  use  any  of  the 
controls  are  required  to  repeat  the  module  that  explains  how  the 
control  is  used. 

After  viewing  the  introductory  videotaped  sequences  and  completing 
the  introductory  training  program,  students  practice  the  skills  by 
interacting  with  the  main  simulation  training  program.  The  student 
spends  about  two  and  one  half  hours  in  a  series  of  thirty  training 
exercises  in  this  program.  These  exercises  are  blocked  (5-10-10-5)  in 
blocks  of  increasing  difficulty,  with  problems  of  approximately  equal 
difficulty  within  a  block.  Appendix  II  gives  a  detailed  narrative 
description  of  a  simple  problem  interaction. 

The  simulation  program  has  the  top  level  structure  depicted  in 
Figure  3.  Note  that  the  system  uses  a  system  of  frequent  checks  of  the 
real  time  clock  for  pacing  the  simulation  rather  than  clock-based 
interrupts.  This  feature  facilitates  straightforward  structured 
programming  in  Apple  Pascal.  The  heart  of  the  simulation  is  the 
Probloop  procedure,  which  constantly  checks  for  student  errors,  for  new 
student  actions,  and  for  new  simulated  events.  The  form  of  the 
exercises  are  determined  by  coded  data  files  that  prescribe  the 
simulated  events  that  are  to  occur  in  each  exercise.  An  exercise  comes 


to  an  end  when  either  the  student  or  the  simulated  enemy  aircraft  win 
or  the  time  alloted  for  the  exercise  in  its  coded  file  is  expended. 


SUMMARY 


We  have  shown  that  dynamic  skill  training  differs  in  important 
ways  from  knowledge  system  instruction.  The  demands  placed  upon 
students'  cognitive  processing  resources  are  different  in  these  two 
types  of  learning.  These  differences  suggest  that  computer  based 
training  of  dynamic  skills  offers  scope  for  research  projects  that  will 
differ  in  methods  and  results  from  conventional  computer  based 
instruction  research.  Issues  for  dynamic  skill  training  in  particular 
need  of  empirical  resolution  include  finding  the  most  effective 
simulation  presentation  techniques  and  the  most  effective  means  for 
providing  instructional  feedback.  Dimensions  of  presentation  dynamics 
to  be  explored  Include  manipulations  of  the  rate  of  simulated  events t 
repetition  techniques,  and  partial-task  training  approaches. 
Instructional  feedback  issues  include  whether  simulation  should 
continue  during  feedback  and  whether  students  or  the  training  programs 
should  have  control  over  the  presentation  of  feedback. 

In  order  to  address  these  and  other  issues  in  dynamic  skill 
training,  a  low  cost  experimental  simulation  training  system  has  been 
developed.  This  system  trains  experimental  students  to  perform  a  task 
similar  in  many  respects  to  that  of  an  air  intercept  controller.  The 
training  program  permits  controlled  differences  in  instructional 
treatments  for  different  subjects.  The  system  is  being  used  as  a  tool 
for  the  empirical  resolution  of  dynamic  skill  training  issues. 
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APPENDIX  1 

Hardware  £04.  Operating  System 

Figure  1  diagrams  the  hardware  of  the  experimental  system  for 
conducting  research  on  computer-based  training  of  real-time  driven 
tasks.  Only  the  major,  system- level  components  are  shown  here. 

The  shaded  elements  are  planned  for  but  not  currently  integrated  into 
the  complete  hardware /software  package.  An  Apple  II  micro-computer  is 
the  heart  of  the  system.  This  computer  uses  the  Apple  Pascal  operating 
system,  within  which  the  simulation- training  programs  have  been 
written.  A  Corvus  10-megabyte  Winchester  technology  hard  disc  drive 
provides  mass  storage  for  the  system.  The  speed  of  this  drive  in 
accessing  and  transferring  data  is  very  important  to  the  success  of  the 
simulation.  The  program  uses  overlaid  segments,  so  fast  disc  transfers 
are  important  to  maintaining  a  realistic  pace  in  the  simulation.  The 
Corvus  disc  drive  provides  a  16K  RAM  buffer  which  typically  holds  the 
next  required  program  segment.  As  a  result,  overlays  from  the  disc 
drive  are  accomplished  very  quickly. 

Three  display  screens  are  used  for  the  simulator  trainer.  The 
first  of  these  is  a  standard  RS-232C-compatible  80  column  CRT  terminal. 
The  keyboard  of  the  terminal  has  been  modified  with  special  key  caps 
corresponding  to  simulated  functions.  This  display  serves  as  the 
alphanumeric  console  of  a  simulated  radar-control  computer.  The  second 
display  screen  is  a  high  bandwidth  green  phosphor  video  monitor.  This 
screen  displays  the  medium-resolution  graphics  output  of  the  Apple  II. 
It  functions  as  a  simulated  radar  screen  in  the  CBIT  system.  The  third 
screen  is  a  color  monitor  used  to  display  videotaped  instructional 
sequences  under  computer  control.  The  computer-based  videotape 


controlling  device  (a  Cavri  V  controller),  which  permits  random-access 
of  videotaped  segments,  also  has  the  capability  of  controlling  a 
videodisc  player. 

Student  interaction  with  the  simulator  trainer  is  by  Mans  of  key 
presses  of  the  specially  coded  key  caps  on  the  CRT  terminal  and  through 
the  use  of  the  linear  joystick  connected  to  the  Apple  II.  In  addition, 
the  system  will  make  use  of  voice  input  and  output  technology  in  future 
experiments.  An  Internal  real  time  clock  permits  accurate  recording  of 
the  times  of  student  actions. 
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APPENDIX  II 
The  Task 

A  narrative  description  can  only  partially  convey  the  nature  of 
the  simulation,  but  one  is  attempted  here.  A  problem  of  relatively  low 
difficulty  might  consist  of  the  following  events.  The  Radar  screen  is 
initially  blank.  A  few  seconds  later,  the  first  radar  sweep  takes 
place,  and  a  blip  appears  in  the  lower  left  corner  of  the  radar  screen. 
The  student  depresses  a  key  labeled  "JOY"  in  order  to  activate  his 
Joystick.  When  the  joystick  is  activated,  a  small  crosshairs  cursor 
appears  on  the  radar  screen.  The  student  manipulates  the  joystick  to 
position  the  cursor  on  the  blip.  A  new  radar  sweep  occurs  (eight 
seconds  after  the  previous  one)  and  the  student  must  reposition  the 
cursor,  using  the  joystick.  With  the  cursor  centered,  the  student 
"hooks"  the  blip  by  depressing  a  symbol  key,  "Cl",  on  the  console 
keyboard.  The  simulated  IFF  function  (Identification:  Friend  or  Foe) 
determines  that  the  symbol  is  appropriate  for  the  blip,  so  the  symbol 
"Cl"  appears  on  the  radar  screen  next  to  the  blip.  The  Joystick  cursor 
disappears,  because  pressing  the  symbol  key  (Cl),  turned  off  the 
joystick.  Another  radar  sweep  takes  place,  and  the  blip  advances  again 
toward  the  upper  right  corner  (NE).  The  symbol  has  been  left  behind 
because  the  simulated  tracking  computer  requires  two  fixes  to  project  a 
course  for  the  symbol.  The  student  reactivates  the  joystick  by 
depressing  the  "JOY"  key,  and  goes  through  the  same  process  of  hooking 
the  blip  with  its  symbol  again.  Now  the  system  draws  a  track  for  the 
blip  and  advances  the  symbol  on  the  projected  path  with  each  radar 
update.  (Note  that  if  the  simulated  pilot  of  the  blip  changes  the 
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course  or  speed,  the  symbol  and  the  track  will  continue  on  the  old 
course  until  the  student  rehooks.) 

Once  the  student  has  hooked  and  rehooked  the  blip,  he  or  she  will 
depress  the  "FUL"  and  "WPN"  keys  to  find  out  aircraft  Cl's  fuel  and 
weapons  status.  These  figures  will  be  posted  in  the  appropriate 
display  area  of  the  simulated  tracking  computer  console  display  screen. 

The  student  then  observes  a  second  blip  appearing  near  the  top  of  the 
screen.  The  joystick  is  activated  and  the  student  attempts  to  hook  the 
new  blip  with  the  "C2"  symbol.  A  beep  from  the  console  Indicates  that 
the  simulated  IFF  function  has  determined  that  C2  is  not  an  appropriate 
designation.  The  student  now  depresses  the  "B1"  symbol  key,  and  the 
symbol  appears.  This  means  that  the  new  blip  is  a  Bogey  (a  bad  guy). 

The  student  reactivates  the  joystick,  and,  after  the  next  screen 
update,  rehooks  the  B1  blip.  Now  that  the  tracking  computer  has  a  fix 
on  the  location  and  course  of  the  Bogey,  an  intercept  can  be  computed. 

The  student  depresses  the  gray  "INT"  key.  On  a  reserved  portion  of  the 
simulated  tracking  computer  console  display  the  student  sees  a  request 
for  the  symbol  of  the  combat  air  patrol  to  be  assigned  to  the 
intercept.  He  depresses  Cl.  The  system  then  asks  for  the  Bogey  to  be 
Intercepted.  The  student  enters  B1 .  The  system  then  displays  a 
recommended  intercept  heading,  say  270  degrees.  The  student  must  now 
send  this  recommendation  to  the  pilot  of  Cl  quickly.  This  is  done  by 
depressing  the  yellow  "INT”  key  and  following  the  system's  directions. 

As  the  two  blips  near  each  other,  the  Bogey  may  take  evasive 
action,  which  the  student  must  respond  to  by  making  the  simulated 
tracking  computer  recompute  intercept  or  attack  headings.  Attack 
headings  must  be  computed  when  a  fighter  is  within  attack  range  of  a 
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Bogey.  The  new  recommendations  must,  of  course,  be  relayed  to  the 
pilot.  Finally,  when  the  student  judges  that  the  fighter  is  within 
firing  range  of  the  blip,  he  directs  it  to  fire  a  missle.  If  the 
fighter  is  within  range  and  on  the  correct  attack  course,  the  Bogey  is 
downed  and  the  student  wins  the  exercise.  The  student  can  lose  an 
exercise  either  by  allowing  the  Bogey  to  shoot  down  the  combat  air 
patrol  or  by  allowing  the  Bogey  to  escape. 

The  exercise  dscribed  was  a  simple  problem  in  that  only  one  Bogey 
and  one  combat  air  patrol  were  present.  The  example  was  further 
simplified  in  that  events  were  regularly  paced  and  the  student  did  not 
make  mistakes  that  might  have  resulted  in  the  presentation  of 
instructional  feedback  messages. 
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